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ABSTRACT 





The focus of this study is to determin when the 








cumulative Cost Performance Index (CPlem) stabilizes for 





different contract characteristics. The CPI is the 





relationship between the budgeted costs for work performed 


divided by the actual costs of work performed. Once the 





CPloum Stabilizes, program managers and analyst are able to 
use this index as a predictor in estimating the final cost 
of the contract. 


The range method and the narrowing interval method were 








used to test for CPloum stability at the 50% complete point. 


For the range method, stability was declared if the range, 








which is the maximum CPIogum value minus the minimum CPIcum 


value over a specified interval, was less than or equal to 








-20% The results for the range method indicated that the 
CPleur was stable at the 50% complete point. Further 











analysis showed that the CPlIem was stable as early as the 


103 to 20% complete point. For the narrowing interval 





method, stability was declared when the variance of the 





CPlem is less than or equal to plus or minus .10 over a 





specified percent complete interval. The results for this 





method indicated that the CPIgm could only be declared 





stable from the 50% complete point. 
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EXECUTIVE SUMMARY 





The focus of this study is to determin when the 








cumulative Cost Performance Index (CPlem) stabilizes for 





different contract characteristics. The CPI is the 





relationship between the budgeted costs for work performed 


divided by the actual costs of work performed. Once the 





CPloum Stabilizes, program managers and analyst are able to 
use this index as a predictor in estimating the final cost 


of the contract. 


Various methods were used in this study. The range 





method was first used to determine if the CPloam stabilized 





by the 50% complete point. For the range method, stability 





was declared if the range, which is the maximum CPIcum value 





minus the minimum CPIeuwm value over a specified interval, was 


less than or equal to .20. The results for the range method 








indicated that the CPlIem was stable at the 50% complete 





point. Further analysis showed that the CPIem was stable as 





early as the 10% to 20% complete point. 


Next the method of least squares was used to determine 
trends in cost performance. The results of this method 
showed that 67% of the contracts included in this study had 
a negative slope, meaning the cost performance worsened as 


the contract progressed. 


For the narrowing interval method, stability was 





declared when the variance of the CPlIum is less than or 


equal to plus or minus .10 over a specified percent complete 





interval. This method is more stringent than the range 








method as it ensures all CPlem values over the interval 





specified are within the plus or minus .10 variance of every 





x1Lil 


other value within the interval. The results for this 


method indicated that the CPIgm could only be declared 








stable from the 50% complete point. 


The categorical evaluation examined the relationship 


between the CPlIem Stabilization points and different 





contract characteristics. This examination allowed 
comparisons to be made among the various types, phases, 
time-frame, and baseline stability of a contract. The 
results showed that fixed price contracts stabilized earlier 
than incentive and award fee contracts. For the different 


phases, contracts in the production phase stabilized earlier 





than contracts in the LRIP and development phases. 
Contracts which began after the A1l2 program cancellation 


stabilized sooner than contracts which started before 








cancellation of the program. Lastly, contracts with stable 








baselines stabilized before contracts with unstable 





baselines. 


The results of this study provide program managers and 





analysts with a solid foundation of CPlIeam stability 








percentages. Knowing when the CPlam may be declared stable 
provides confidence in the estimated cost at completion and, 
if a cost overrun is projected, the likelihood that a 


contractor can recover. 


Xiv 


i ae INTRODUCTION 


A. GENERAL ISSUE 








Cost estimation of Department of Defense (DoD) weapon 








systems iS an enormous yet vitally important undertaking. 








With the cost of major weapon systems skyrocketing and more 


and more systems fighting annually for a piece of the 








budget, the DoD needs to ensure it is spending its money 











wisely. The DoD uses the Earned Value Management System 
(EVMS) as one management tool to accomplish this task. By 


tracking specific metrics derived from a contractor’s EVMS, 











the DoD is able to gauge the value it is getting for the 


money it is spending. One of these metrics is the Cost 











Performance Index (CPI). CPI is simply the relationship 








between the budgeted costs for work performed divided by the 








actual costs of work performed. Many experts within the 
acquisitions community declare that this is the most 


critical metric provided by Earned Value Management (EVM). 








However, a single CPI value provides only a snapshot in 





time of how a contract is performing. The true benefit 




















comes from tracking the cumulative CP (CPleum) aS a contract 





progresses from start to finish. By tracking the CPleum, 
cost performance trends are easily recognized. Early 
detection of downward cost performance trends alerts 
management that changes need to be made quickly if the 


contract is to be completed within the budgeted amount. 








In order to get the most value from the CPlIoum, we must 


first be able to declare that it is stable. The CPIcum 18 








declared stable when the variance over a specified interval 


1 





is less than plus or minus .10. Once the CPloum is declared 


stable, analysts are able to produce fairly accurate 








estimates for the final cost of the contract. 





Unfortunately, no one can be 100% certain that a contract is 





stable until it is completed, at which point the final cost 


is definitely known. Many experts have devised their own 








heuristics to determin when CPlIeum stability occurs. One 





method used is to declare the CPlIoam stable six months after 
the contract is awarded. Another method is to declare 


stability after the contract is 20-25% completed. Whether a 





time elapsed or percent complete method is used is usually 





based on the analysts’ specialty and years of experience. 


Although the experts have different opinions on when 





CPloum Stability occurs, there is agreement on the benefits 








of a stable CPTIeun. As previously stated, a stable CPlom 











provides confidence in the estimate of the final cost. Tt. 


provides early warning of potential cost overruns. 





No longer must management wait until all the funds 





have been spent to determine that additional 
budget will be needed in order to complete the 
full scope of a given project. The CPI thus 





represents the project manager’s “early warning 
signal” and is perhaps the most compelling reason 
why any project should employ some form of Earned 
Value. (Fleming & Koppelman, 2005) 





Other benefits include the ability to evaluate the 
contractor’s internal management system and planning 


process. 





The benefits of knowing when the CPIem is stable are 


the “why” of this study. The “what” is to conduct an 





empirical study, whose results will provide program managers 





with a tool in declaring when the CPlIeum is stable. 


2 


B. RESEARCH PROBLEM 


The problem for this study is to 





CPloum Stabilizes for contracts of differe 





determin when the 





nt categories. The 


four categories to be examined are type, phase, stability, 





and year. These different categories will be explained in 


greater detail later in this chapter. 


Cc. HYPOTHESIS 


The null hypothesis is that stability occurs when a 





program is greater than 50% complete. Two different methods 
will be used to examine stability. The two methods are the 





method for which 





range method and the narrowing interval 


stability will be defined differently for each method. The 








two methods and two definitions for 














discussed in Chapter 





D. SCOPE OF RESEARCH 


All data for this thesis came 





stability will be 





from the Defense 








Acquisition Management Information Retrieval (DAMIR) system 


which is provided and maintained by the Office of the Under 








Secretary of Defense for Acquisition, Technology and 


Logistics (OUSD (AT&lL)). The database 








schedule performance for Major Defense A 





contains cost and 


cquisition Programs 


(MDAP) dating back to 1971. The information stored in the 





database originates from Defense Acquisition Executive 








Summaries (DAES). The DAES are quarter 





ly reports sent to 











OUSD (AT&L) by Program Managers (PM) 





storage. These PMs represent each of t 





military. The actual DAES report is prep 


information provided by the contractor 


for analysis and 
he branches of the 


ared by the PM from 





: The contractor 


prepares a monthly report called the Cost Performance Report 
(CPR) containing the current month’s program cost and 
schedule performance. The information available spans a 


diverse set of programs from ships, planes, and tanks, to 





radios, software, and support equipment. The contracts also 


represent programs from the various lifecycle cost and 














milestone phases, such as, Development (DEV), 
Demonstration/Validation (DEM/VAL) and Construction. They 
also include Low Rate Initial Production (LRIP) and Full 














Rate Production (FRP) and the different types of contracts 





currently being awarded. The list includes Firm Fixed Price 


(FFP), Fixed Price Incentive F (FPIF), Cost Plus Incentive 

















Fee (CPIF), Cost Plus Award Fee (CPAF), and Cost Plus Fixed 


Fee (CPFF). 


E. ASSUMPTIONS AND LIMITATIONS 


The major assumption made in this thesis is that the 














data drawn from DAMIR is accurate. By DoD’s requirement for 





contractors to comply with EVMS criteria, it is reasonable 








to expect that the data provided is reliable. Contractors 





of MDAP are required to be EVMS criteria-compliant. 


The first major limitation is that not all contracts 





contain the necessary data to be analyzed. To conduct the 
necessary analysis, we required that a contract must have 


data from the 20 through 85 percent interval of completion. 





For many of the contracts included in the database, the full 
interval is not reported. Some only include data starting 
from beyond the 20 percent completion point while others do 


not include data covering the later portion of the contract. 





Of significant importance is that programs which are 


cancelled prior to the 80 percent completion point will not 
4 


be included in this study. Many of these cancelled programs 
are due to poor cost and schedule performance which would 
almost certainly prove to be unstable beyond the 50 percent 


completion point. 


The second limitation is that some of the contracts 





which span the 20 to 80 percent completion interval are 


missing values within this range. This limitation is 





overcome uSing interpolation to fill in the missing values. 





The final limitation is that I was not granted access 





to the Army and Air Force data in a timely manner for their 





data to be included in this study. Fortunately, there were 





181 contracts from the Navy which met all the requirements 





needed to be included in this study. 


Before we can begin to analyze these 181 contracts, an 
understanding of some of the finer points of EVMS is 


required. The next chapter will provide a thorough 





background of the EVMS and the benefits it provides. 
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II. LITERATURE REVIEW 


A. BACKGROUND OF EARNED VALUE 


Since the 1960s, Earned Value Management (EVM) has been 


used as a tool to allow program managers and decision makers 





to have visibility into technical, cost, and schedule 
progress. The implementation of an earned value management 
system (EVMS ) is a recognized function of program 
management. It ensures that cost, schedule, and technical 





aspects of a contract are truly integrated. 


The concept of earned value began over a century ago by 
industrial engineers in American factories. By converting 


“planned industry standards” into “earned standards” and 








then relating them against “actual hours,” thes ngineers 
began to focus on true cost performance (Fleming & 
Koppelman, 1994). In comparing actual hours against earned 





standards, these early industrial engineers defined what 





today is termed “cost variance.” The approach just 


described is the foundation of EVM. 








The earliest system within the DoD recognized as using 


the earned value concept was the Program Evaluation and 





Review Technique (PERT). PERT was developed in 1958 by the 


U.S. Navy as a network-scheduling device. The PERT approach 





attempted to simulat th development planning of a new 








project in the form of a logic flow diagram, and then to 





assess the statistical probability of actually achieving the 





plan (Fleming & Koppelman, 2005). Due to a combination of 


insufficient computer resources, complexity, and rigorous 


implementation requirements, PERT was essentially abandoned 
as a management tool by the mid-1960s (Fleming & Koppelman, 
1996). 





About the time the Navy’s PERT concept faded from the 





spotlight, the Air Force appeared with its own version of 


earned value. Within the group of developers of this new 





approach were some of the very same people who designed th 

PERT approach. With PERT’s misgivings still fresh in their 
minds, they quickly agreed that they would not impose any 
specific “management control system” on private industry 
(Fleming & Koppelman, 2005). This new approach set 35 


criteria which established the minimum requirements of an 





acceptable project management system. In abiding by these 
35 criteria, industry simply needed to respond to basic 
questions based on sound project management principles. 


Rather than directing industry on how to manage, these 





criteria ensured industry used ffectiv and measurable 





management practices. The first use of these criteria was 
in 1963, for the U.S. Air Force’s Minutemen Program (Abba, 


1997). Contractors were receptive of this new approach as 





they were given flexibility to tailor their individual 








management systems in order to meet contract requirements. 











In 1967, the Department of Defense, realizing the 





usefulness of an EVMS, published DoD instruction 7000.2 








establishing the Cost/Schedule Control Systems Criteria 
(C/SCSC). Similar to the Air Force’s system, C/SCSC was a 


set of criteria that a contractor’s internal management must 





meet. Thus, C/SCSC was not a management system, but a guide 


of the minimum standards required. Although C/SCSC reigned 








for the next 30 years as the DoD’s standard for contract 


management, it was never fully embraced by private industry. 


The major reasons for its lack of acceptance wer XCESSIV 





checklists and paperwork, specialist acronyms, and rising 
administrative costs due to over-implementation of the 
criteria (Antvik, 1998). Some viewed the earned value 


methodology as excessive “bean counting” (Abba, 1997). With 





private industries’ obvious disdain for the current system, 


earned value was ready for reform. 


Reform came about in the mid-1990s as part of a 














National Defense Industrial Association (NDIA) initiative, 





in which private industry took a proactive role in 
redeveloping the minimum criteria required. The initiative 
consisted of modifying the original 35 C/SCSC criteria into 


32 straightforward guidelines (Fleming & Koppelman, 2005). 





In 1997, these 32 revised criteria were published as part of 








DoD Instruction 5000.2R. The most significant difference 





between this system and its predecessors was the “buy-in” by 





private industry. In effect, ownership of earned value was 








transferred from DoD to- private industry. Another 


significant distinction was that this system was viewed as a 





project management tool as opposed to a financial management 








one. Shortly after the publication of DoD Instruction 











5000.2R, the EVMS was adopted by the NDIA as the American 




















National Standards Institute (ANSI/EIA) Standard #748, in 





June 1998. 


B. EARNED VALUE DEFINED 


Earned value management iS a project control 
process based on a structured approach to 
planning, cost collection and performance 
measurement. It facilitates the integration of 
project scope, time and cost objectives, and the 


9 








establishment of a baseline plan for performance 


measurement. (Association for Project Management, 
2006) 
The thr ssential features of EVM are (1) a project plan 





that identifies the work to be accomplished, (2) a valuation 





of planned work, and (3) pre-defined metrics to quantify the 


accomplishment of work. 


Cc. EVALUATION OF A CONTRACTOR’S EARNED VALUE MANAGEMENT 
SYSTEM 











DoD Directive 5000.4 charges the Cost Analysis 








Improvement Group (CAIG) with developing an independent cost 








estimate for Acquisition Category ID programs, pre-Major 








Defense Acquisition Program projects approaching formal 








program initiation as a likely Acquisition Category ID 











program, and Acquisition Category IC programs when requested 

















by the Under Secretary of Defense, AT&L. CAIG evaluates the 


EVMS used by the contractor to ensure efficient and accurate 








implementation. A technique which CAIG uses in its 





evaluation process is to review the contractor’s Cost 


Performance Reports (CPRs). 


D. CPR ANALYSIS 


The CPR provides the status of progress of the 


contract. The key data elements provided in the CPR are: 


° Actual Cost of Work Performed (ACWP) - The costs 
actually incurred and recorded in accomplishing 
the work performed within a given time period. 








° Budget At Completion (BAC) - The Contract Budget 
Base less Management Reserv 








° Budgeted Cost for Work Performed (BCWP) - The sum 
of the budgets for completed work packages and 
completed portions of open work packages, plus the 
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applicable portion of the budgets for level of 
effort and apportioned effort. 





e Budgeted Cost for Work Scheduled (BCWS) - The su 
of the budgets for all work packages, plannin 
packages, etc., scheduled to be accomplishe 
(including in-process work packages), plus’ th 
amount of level of effort and apportioned effor 
scheduled to be accomplished within a given tim 
period. 











oOortonawdgs 





e Contract Budget Base (CBB) Th negotiated 
contract cost plus the estimated cost Ot 
authorized unpriced work. 




















e Estimate At Completion (BAC) —- Actual direct 
costs, plus indirect costs allocable to the 
contract, plus the estimate of costs (direct and 

















indirect) for authorized work remaining. 

e Management Reserv (MR) - An amount of the total 
allocated budget withheld for management control 
purposes rather than designated for the 





accomplishment of a specific task or set of tasks. 


From these basic data elements, performance metrics are 
deduced. There are many metrics which can be deduced from 


the data provided in the CPR; however, the only metric 





required for this study is the Cost Performance Index (CPI). 








Two other metrics ar th Schedul Performance Index (SPI) 














and the To Complete Performance Index (TCPI). These two 


metrics are mentioned only to provide information on how 





they are used with the CPI. 





The SPI indicates the ability of the contractor to 





control the project schedule. The SPI compares the budget 
for those tasks scheduled to be accomplished as of the 
status date with the budget for the work that was actually 
accomplished as of that date (Fleming & Koppelman, 1996). 








The formula for computing the SPI is: 








SPI = BCWP / BCWS (1) 
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SPI can be non-cumulative or cumulative based on the data 














used. For illustrative purposes, assume that through the 
sixth month of a contract, BCWP = $95,000 and BCWS = 
$100,000, then the cumulative SPI = BCWP/BCWS = 0.95. The 


take-away is that only 95 percent of the work scheduled was 


accomplished, thus the contractor is slightly behind 





schedule (5%). An index of 1.0 means the contract is 


progressing exactly as planned from a scheduling point of 





view. If the index is greater than 1.0, the contract is 








progressing ahead of schedule. If the SPI varies too much 


from the 1.0 baseline, further investigation would be 





required to determin th cause for the variance from 
schedule. Large variances could be caused by inclement 
weather for an outdoor project, or could be the result of 
poor planning by the contractor, a problem with 


manufacturing, etc. 





Similar to the SPI is the CPI. Like the SPI, the CPI 




















can be cumulative or non-cumulative. The CPI indicates the 


ability of the contractor to control cost, and compares the 











budget for those tasks that have been accomplished with the 





actual cost of accomplishing the tasks (Fleming & Koppelman, 





1996). The formula for computing the CPI is: 
CPI = BCWP / ACWP (2) 





From our earlier example, assume that at the end of the 


sixth month our ACWP = $110,000. Our CPI would then egual 











$95,000 / $110,000 = .864. The takeaway here is that the 
contractor is only earning around 86 cents of value for 
every dollar spent. At this point the contractor in our 
example is performing behind schedule and over planned 
budget. An index of 1.0 means that the contract is 


performing as planned from a cost perspective. An index 
12 


greater than 1.0 means the contract is currently performing 
under the planned budget. Unfortunately, the majority of 
contracts are similar to the example above, in that they are 


generally both behind schedule and over budget. 


The final index to be defined informs the project 





manager of how well the contractor must perform throughout 
the rest of the contract to finish within the planned 


budget. This metric is the TCPI and the formula is: 





TCPI = (BAC — BCWP) / (BAC — ACWP) (3) 





Continuing with the example presented and a BAC = $200,000, 





the TCPI = 1.167. To finish within the planned budget, the 














contractor must perform at an overall CPI rate greater than 





1.167 for the remaining portion of the contract. As the 








contractor has performed at a CPI below 1.0 thus far, it is 
unlikely that the contractor will be able to meet the 


planned budget. 
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III. METHODOLOGY 


A. THE DATA 


All data used in this thesis was extracted from the 











Defense Acquisition Management Information Retrieval (DAMIR) 








system. In order for a contract to be included in this 
study, data for the contract needed to be reported at or 
below the 20 percent completion point as well as at or above 


the 80 percent completion point. The actual data elements 








retrieved throughout this interval were the CPlam values. 





Also captured wer th program name, contract subject, 
program phase, contract type, year contract began, year 
contract ended, and stability of the baseline. These 


elements of information will be used to categorize contracts 





in order to perform comparative analysis. 


One hundred eighty-one contracts from 48 different 


programs met the criteria for inclusion. The 48 programs 





are representative of the major investments made by the Navy 








over the last 30 plus years. Samplings of these investments 


include the purchases of aircraft (from the F-14 through the 





V-22), construction of ships, ammunition research and 





























development, and software development. The contract types 
represented wer Firm Fixed Price (FFP), Fixed Price 
Incentive F (FPIF), Cost Plus Incentive F (CPIF), Cost 
Plus Award Fee (CPAF), and Cost Plus Fixed Fee (CPFF). The 
program phases represented wer Development (DEV) , 














Demonstration/Validation (DEM/VAL), Construction, Low Rate 














Initial Production (LRIP), and Full Rate Production (FRP). 
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B. METHOD OF ANALYSIS 


Three different methodologies will be used to analyze 
the data. The first two methods will be used to validate 


the results attained by Captain Scott Heise in his thesis 








titled, “A Review of Cost Performance Index Stability” 
(1991). A third, more stringent method will then be used to 
analyze the data. The three methods are: 

Ing Range Method 

Qi Least Squares Method 

Bie Narrowing Interval Method 











First, all 181 contracts will be used to test the 





hypothesis to be stated. Then the contracts will be tested 


on various categories, including by type, phase, year, and 





stability. Finally, the contracts will be evaluated by 





combinations of the categories. 





Ly Hypothesis 


For the first and third methods, the hypothesis to be 








tested is that the CPlIom stabilizes by the 50 percent 





completion point. The second method to be presented will 


not have an associated hypothesis. 


2. CPI Calculations 








The CPleum was extracted directly from the DAMIR system. 











The formula used in computing the CPlIcum was: 


CPleum = BCWPounm / ACWP cum (4) 








The CPlewm was recorded for the following percent complete 


points of each contract: Bye 0, -L5;,..-20;,° 25, -30;, -40;,- 50,5. 60; 





70, $80, 90, 95, and 100, if these points were available. 


Recall that to be included in the database for this thesis, 
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a contract has to begin reporting at no later than the 20 











percent level and end reporting at no earlier than the 80 











percent level. In instances where contracts met the 


requirements just stated but were missing one or more 




















intermediate CPIoum values, the CPIgym values were interpolated 





for those points. For example, if a contract had a CPloum of 


-97 at the 30 percent complete point and .95 at the 50 








percent complete point, the CPIem used in the analysis for 





the 40 percent complete point would be .96. 


3. Percent Complete Calculations 


Percent complete is the ratio of the amount of work 





accomplished to date to the amount of work planned for the 





total contract. Like the CPlIem, percent complete was 














extracted directly from the DAMIR system. The formula used 








in DAES reporting to compute percent complete is: 





PERCENT COMPLETE = BCWPeum / (CBB — MR) (5) 


where 


CBB = Contract Budget Base 





MR = Management Reserv 














If you recall from Chapter , CBB - MR = BAC. Thus, a more 





compact formula is: 





PERCENT COMPLETE = BCWPcum / BAC (6) 


where 
BAC = Budget At Completion 


Using these two formulas to compute percent complete 





does have a drawback, however, if any new effort is added to 
a contract, the CBB, and thus the BAC, increases. This in 


turn causes a decrease in the percent complete calculated. 
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If the amount of effort added is substantial, the 


percent complete may actually decline from one cost 





performance period to the next. Contracts exhibiting this 





behavior are identified as having an unstable baseline. 


Earlier in this chapter, it was mentioned that stability of 














the baseline was extracted from the DAMIR system. In this 





study, a contract baseline will be declared as unstable if 





either (1) the percent complet decreases between any two 
consecutive cost performance periods or (2) the contract 
undergoes an “over target baseline.” Over target baseline 


(OTB) is defined as “a project baseline that results from 





the acknowledgement of an overrun, and actually incorporates 








the forecast overrun into the performance baseline for the 





remainder of the work” (Fleming & Koppelman, 2005). Similar 
to a contract which has new effort added, a contract which 
undergoes an OTB will have an increase in the CBB and BAC, 
and a corresponding decrease in the percent complete. A 


contract which undergoes an OTB will be declared as having 





an unstable baseline regardless of whether the percent 





complete actually decreases between any two consecutive cost 





performance periods. The reader should be cautioned not to 





confuse the stability of the baseline with CPI stability. 





Stability of the baseline is one of the categories which 


will be analyzed. 


Another calculation which has been used to determine 





percent complete substitutes monthly BAC and final BAC in 








equation (6). In one study which used this approach, it was 





found that the number of contracts with stable CPIS was 





identical using both approaches (Payne, 1990). The drawback 


to this approach is that the final BAC must be known. For 











the majority of contracts available in the DAMIR system, the 
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final BAC is not reported. Thus, only the first approach to 





calculate percent complete will be used in this study. 


4. Range Method 








In his thesis, Heise considers the CPlIoem stable when 





the CPIeum does not vary more than plus or minus .10 (Heise, 





1991). He uses the range method to test for CPIleum 





stability. To determin th range, the minimum CPlem is 











subtracted from the maximum CPIogun located in the percent 





complete interval of interest. A contract is then 
considered stable if the range is less than or equal to .20. 


These procedures will be followed for the first method in 


























this study. The interval of interest is between the 50 
percent complete point and the final reported percent 
complete point. Although the 50 percent complete point is 
the parameter set in the hypothesis, analysis will also be 


conducted for the 40, 30, 20, 10, and 5 percent complete 


points. 


5. Confidence Interval Calculations for the Range 
Method 


The 90, 95, and 99 percent confidence intervals will be 


calculated for the mean of the ranges. The large-sample 





method for determining confidence intervals will be used. 





The large-sample method confidence interval is calculated 


using the following equation: 


UT) 


where 


x = the sample mean 
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Zag SS Le twOmine Ll: 2 -er ities Jk arelae 
Ss = the sample standard deviation 
n = the sample size 


This method is appropriate when the sample size is greater 





than 40 as the standardized variable has approximately a 





standard normal distribution. Using 40 vice 30 as a rule of 
thumb for the Central Limit Theorem is due to the additional 


variability introduced by using the sample standard 





deviation in place of the population standard deviation 





(Devore, 2004). The level of confidence for the confidence 








interval indicates the number of times out of 100 that 








computed confidence intervals are expected to contain the 


true mean. 


6. Least Squares Method 


Analysis using the least squares method will be 


conducted to identify trends in cost performance. The first 








method (range method) focused on the variance of the CPToun. 








With the first method, instability was declared if the 





differenc between the maximum and minimum CPlIoem values 











within the specified percent complete interval was greater 
than .20. By this definition, instability was declared 
whether the .20 breach occurred in the upward or the 


downward direction. The “upward direction” should be taken 








as the CPlIem iS increasing (improving) as the percent 








complete is increasing. It is theoretically possible that 
a majority or all of the breaches occurred in an upward 


direction (we will see that this was not the case). 





Taking the mean of the least squares will provid videnc 








as to the upward or downward trend of the CPleun.- In 
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addition to the direction, this method will also provide the 


magnitude of the cost performance trend. 


The method of least squares consists of finding the 





best-fit line to observed data points (CPIom values in this 





study). The best-fit line is the line that minimizes the 





sum of squared vertical deviations between the estimated 


line and the plotted values (Devore, 2004). The parameters 





which describe this best-fit line are the slope (S) and the 


intercept. It is the slope which indicates the magnitude 








and direction of the line, and thus provides us with the 
cost performance trend. The least squares equation used to 


estimate the slope is: 





ee n> X,Y; Yayo y;) 





nye (x) (8) 
where 
x = the percent complete 
y = the CPlom value for the period investigated 
n = the number of cost performance periods investigated 


The equation for the intercept is not provided as it will 


not be used in this study. 


7. Narrowing Interval Method 





The range method compares the difference between the 





maximum and minimum CPIegm values over a specified interval 
against a set parameter which defines stability (.20 in this 


study). The narrowing interval method works backward from 





the last reported CPlaum value re-computing a narrowing 


interval which is determined by subtracting .10 from the 
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highest value and adding .10 to the lowest value seen thus 


far. This backwards moving process continues until the 





preceding complete point falls outside of the interval just 
calculated. Stability is declared for all percent complete 


points up to but not including the point which fell outside 








of the interval. At all other percent complete points, the 





CPleum is declared unstable. 


The narrowing interval method will be applied to the 





following fictional data to help clarify the process: 











Table l. Fictitious CPlewm Data 
% 10 20 30 40 50 60 70 80 90 100 


Complete 



































CP locum 1.1 )1.07]1.03) .98 | .97 | .95) .94 | .88 |] .89 | .90 


We start by computing an interval which is .10 + the 








last reported CPlIem value. The last reported value is .90 








at the 100 percent complete point. Thus, the interval is 


LeeOy 00] # The preceding CPlIem value is now checked to 





see if it is within the interval. For this example, the 90 








percent complete point has a CPlIem value of .89 which is 


within the interval. A new interval is calculated by 





subtracting .10 from the maximum CPlIeum and adding .10 to the 














minimum CPlem values seen thus far. The new interval is .90 
- .10 to .89 + .10 or [.80, .99]. The preceding CPIleum value 
is checked to see if it is within this new interval. For 
this example, the preceding CPlom value is within the 








calculated interval until the 40 percent complete point is 
reached. The calculated interval at the 40 percent complete 


point is .98 - .10 to .88 + .10 or [.88, .98]. Now, the 
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CPlom value of the preceding complete point is 1.03, which 








falls outside of the calculated interval. Thus, the CPlowm 





in this example would be declared stable for all complete 





points from 40 to 100 percent; and for the complete points 





from 10 to 30 percent, the CPIogum would be declared unstable. 














Each time a new interval is calculated it will either 
decrease in size or remain the same, hence the name of the 
method. Unlike the range method in which the interval is 


always .20, the interval shrinks each time a new maximum or 








minimum CPlem value is observed in the backward direction. 


The shrinking interval of this method makes it more 








stringent in declaring the CPlIleam stable as compared to the 





range method. 


8. Confidence Interval Calculations for the Narrowing 
Interval Method 


After applying the range method; 90, 95, and 99 percent 
confidence intervals will be calculated for the mean of the 


ranges. Calculating the mean of the intervals following 





application of the narrowing interval method would not 





provide any useful information. Instead, the percentage of 








stable contracts at each complete point will be computed and 
then, corresponding 90, 95, and 99 percent confidence 


intervals will be calculated for the determined percentag 





of stable contracts. The formula for computing the 


percentage of stable contracts is: 


% Stable =~ (9) 
n 
where 
x = the number of stable contracts 
n = the sample size 
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The formula for computing the appropriate confidence 


interval is: 











ae ae 
Cl= ; (10) 
1+(Z5,,)/n 

where 

p = the sample fraction of stable contracts 

GS USP 

Liven the two-tail g-cribical wealuse 

n = the sample size 
Equation (10) produces a confidence interval for a 





population proportion (Devore, 2004). 
9. Categorical Evaluation 


This section describes how the different categories 


will b determined and evaluated. First, all methods 





previously described will be applied to the 181 contracts. 








The contracts will then be categorized by the type of the 








contract. Some of the contracts in the DAMIR system were 





aw 








listed having two types, such as, “CPIF/CPFF” or “FFP/FPIF. 


For categorizing in this thesis, contracts will be evaluated 





in one of these four groupings: 











e FPIF 
e CPEF 
e CPIF & CPAF (this grouping includes CPIF, CPAF, 


and CPIF/CPAF combinations) 





° FFP & FFP Combinations (this grouping includes FFP 
and any combination which has FFP as one of the 
types listed) 





Of the 181 contracts, only six do not fit into these 





groupings. The next categorization will be executed on the 
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program’s phase during the contract. Data will be grouped 





into three phases: Development, LRIP, and Production. The 





development grouping will be composed of contracts with the 









































following description: Engineering and Manufacturing 
Development (EMD), Full-Scale Engineering Development (FSED, 
Systems Integration, Design Development, and 
Demonstration/Validation. The production grouping will 
include contracts with the following description: Buys, 


Construction, Production, and Follow-on Production. 








Next, sorting will be performed based on the timefram 
of the contract in reference to the A-12 program 
cancellation. The sample will be divided into the following 


three time periods: 








e Pre-Al2, which are those contracts completed by 31 
Dec 1991. 
e Transitional, which are those contracts which 








started prior to 31 Dec 1991 but were completed 
after 31 Dec 1991. 





e Post-Al2, which are those which started after 31 
Dec 1991. 








The purpose for conducting this sorting is to determine if 


the A1l2 program cancellation and the subsequent acquisition 








reforms improved defens cost performance (Christensen & 
Templin, 2002). The Al2 program was cancelled in January 


1991 due to excessive cost overruns and schedule slippages. 








Prior to the cancellation of the program, an administrative 





inguiry was conducted by the Navy’s Chief Inquiry Officer, 


Chester P. Beach. The administrative inguiry was conducted 








to determine facts and circumstances surrounding the 


variance between the current status of the Al2 Program and 





representations made to the Office of the Secretary of 
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Defense (OSD) (Beach, 1990). Following the cancellation of 





the program and based on the recommendations included in the 
administrative inquiry, numerous acquisition reforms were 


mandated. For a more in-depth look into this topic, see 











“EAC Evaluation Methods: Do They Still Work?” by Dr. David 








Christensen and Dr. Carl Templin. 


The final grouping will be based on the stability of 





the contract’s baseline. Stability of the contract’s 








baseline was thoroughly discussed earlier in this chapter in 








the section titled, “Percent Complete Calculations.” The 





contracts will be grouped as either stable or unstable. 








Analysis will be done on each of the individual 
groupings described in this section. Analysis will then be 
done on combinations of the groupings. The 181 contracts 


will eventually be divided and analyzed in 58 groups of 





various combinations based on the contract descriptions just 








discussed. 
Cc. JUSTIFICATION OF METHODOLOGIES 

The range method as previously discussed is well-suited 
for examining the variance of the CPTeun. However, the range 








method has a slight flaw if stability of the CPIgm is 





defined as being plus or minus .10 from its value at a 





specified percent complete point. With this definition, the 








plus or minus .10 range should be centered on this specified 
percent complete point. The range method does not anchor 


the range, but rather selects the most favorable .20 range 








for the data being analyzed. This flaw is more easily shown 
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by examining the fictional data presented earlier. The same 


data are presented her 





Table 2. Fictitious CPIgm Data 








% Complete 10 20 30 40 50 60 70 80 90 | 100 








CPleum Led DOP | O32) 98 OP SOS Gd: | BB 689° 290 









































Using the range method to test for CPlIem stability from 








the 20 percent complete point, the maximum CPIlIem is 








determined to be 1.07 and the minimum CPloam is determined to 


be .88. Subtracting .88 from 1.07 results in a range of .19 








which is less than .20, thus, the CPlem is declared stable 





at the 20 percent complete point. However, the definition 








of stability for CPlIem was not met. The CPIcum value at the 
20 percent complete point did vary by more than plus or 


minus .10 from this point, in fact, it varied by minus .19 








[=1.07 - .88]. Although the range method fails if it is 





used to verify the definition of CPlam stability as being 





plus or minus .10, it succeeds if it is used to verify CPTeum 


stability as being within a set parameter (.20 in this 





study) over a specified interval. The range method will be 





used in this study to verify stability using the latter 


definition just presented. 





Unlike the range method which attempts to verify CPTleun 


stability, the least squares method investigates trends of 








the: CP laine The mean of the slopes identifies the direction 


and magnitude of the average contract. 


The narrowing interval method overcomes the flaw 








described in using the range method to verify stability of 
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the CPlIem defined as being within plus or minus .10 of a 


specified percent complete point. This method ensures that 











the CPlIeum value at each percent complete point is within 








plus or minus .10 of the CPIem value at every other percent 








complete point in the interval defined. Since this method 





ensures all points are within the plus or minus .10 span, it 











is more stringent in declaring the CPlIeum stable, which will 


be seen in the next chapter. 


Throughout this paper, all discussion regarding the 








cost performance index has referred to the CPleun.- A 





thorough analysis of the stability of the non-cumulative CPI 








was conducted by Heise. In his study, Heise found that the 








non-cumulative CPI was far less stable than the CPIoum 














(1991). Intuitively, cumulative CPI values should tend to 











stabilize sooner than the non-cumulative CPI values due to 
their cumulative nature. Therefore, stability of non- 


cumulative CPI values will not be examined in this study. 








Another method — not examined in this study — is the 





method used by Christensen and Templin (2002). In their 














paper, they compared the final CPlIem against the CPlcoeum at 





the 20 percent complete point. If the absolute difference 








between the two values was greater than or equal to .10, the 














CPloum was declared unstable. This method only looked at the 
end points of the interval. Thus, it is possible that 
intermediate points within the interval breached the .10 
threshold. As the narrowing interval method checks the .10 











threshold for all points within the specified interval, this 


drawback is overcome. 
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IV. RESULTS 


A. RANGE METHOD 


The range method was used to validate the results 





achieved by Heise, and the full results are listed in 


Appendix B. The range value indicates the difference 








between the maximum and minimum CPIeum values observed from a 





specified percent complete point to the last reported 





complete point. A summary of the results is provided in 


Table 3. The table includes the number of contracts 








analyzed, the number of contracts having a stable CPTeum 





(range less than or equal to .20), th percentag of 


contracts having a stable CPloum the maximum range observed, 





the minimum range observed, the mean of the ranges, and the 


standard deviation of the ranges. 





Table 3. Summary of the Range Method CPlem Results 


Percent Complete 
































5% 10% 20% 30% 40% 50% 
Number of Contracts 103 141 180 18] 18] 18] 
Number Stable 719 12] 167 174 175 175 
Percent Stable 717% 86% 93% 96% 97% 97% 
Maximum Range 1L3.0:5 0.71 535 0.34 0.32 0.31 
Minimum Range .02 .02 0 0 0 0 
Mean of the Ranges 0.172 0.132 0.104 0.088 0.075 0.063 











Standard Deviation 0.164 0.097 0.066 0.058 0.053 0.052 





Table 4 presents the results attained by Heise using 





the range method on 155 contracts taken from the Defense 





Acquisition Executive Summary (DAES) database. The 155 
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contracts represented contracts from all branches of the 








armed services. The results ar presented in the same 
format as Table 3. One exception is that Heise uses the 0% 








complete point vice the 5% complete point as was used in 
this study. The 5% complete point was used in this study 
due to the limited number of contracts which had data 


reported below this point. 





Table 4. Heise’s Summary of the Range Method CPIceum Results 
(Heise, 1991, p. 33) 


Percent Complete 






































0% 10% 20% 30% 40% 50% 
Number of Contracts 110 LBZ 15:5 1S 15:5 1.55 
Number Stable 59 116 134 14] 150 153 
Percent Stable 54% 76% 86% 91% 97% 99% 
Maximum Range 1.243 0.644 0.434 0.364 0.312 0.299 
Minimum Range 0.017 0.017 O.017 O.007 0.003 0.003 
Mean of the Ranges 0.262 0.145 0.115 0.096 0.081 0.069 
Standard Deviation 0.213 0.103 0.078 0.068 0.056 0.051 








Comparing the “percent stable” and the “mean of the 
ranges” rows from the two tables, the results appear to be 
very similar for two different sets of data. There is some 


overlap in the data used; however, the data from contracts 





used in both studies account for less than 10% of the 





overall data. Heise concludes his study stating that the 





CPloum is stable at the 50% complete point and provides 





evidence that it stabilizes as early as the 20% complete 





point (Heise, 1991). The results of this study support this 


conclusion using the range method and defining stability 





when the range is less than or equal to .20. 
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B. CONFIDENCE INTERVAL FOR THE MEAN OF THE RANGES 


The mean of the ranges was presented in the previous 





section. In this section, the 90%, 95%, and 99% confidence 
intervals for the mean of the ranges will be presented. The 





confidence intervals show the number of times out of 100 





that the true mean is expected to be within the calculated 





intervals. For the 90% confidence level, it is expected 














that if confidence intervals were calculated for 100 random 


samples (mean of the ranges in this study) taken from a 





population, 90 of these calculated confidence intervals 
would contain the true mean. As the confidence level 


increases, so does the confidence interval. 


Tables 5 through 7 show the calculated confidence 





intervals for the 90%, 95% and 99% confidence levels, 


respectively. The tables list the number of contracts 








analyzed, the mean of the ranges, the standard deviation of 








the ranges, the two-tail Z eritiead value, and the 





calculated upper and lower limits. 





Table 5. 90% Confidence Interval for the Mean of the Ranges 


Percent Complete 


























5% 10% 20% 30% 40% 50% 
Number of Contracts 103 141 180 181 181 181 
Mean of the Ranges 0.172 0.132 0.104 0.088 0.075 0.063 
Standard Deviation 0.164 0.097 0.066 0.058 0.053 0.052 
Z Critical Value 1.645 1.645 1.645 1.645 1.645 1.645 
Upper Limit 0.198 0.145 0.112 0.095 0.081 0.069 
Lower Limit 0.145 0.118 0.096 0.080 0.068 0.057 
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Table 6. 95% 




















Confidence 





Percent Complete 


Interval for the Mean of 















































the Ranges 








5% 10% 20% 30% 40% 50% 
Number of Contracts 03 141 180 181 181 181 
Mean of the Ranges 0.172 0.132 0.104 0.088 0.075 0.063 
Standard Deviation 0.164 0.097 0.066 0.058 0.053 0.052 
Z Critical Value 1.960 1.960 1.960 1.960 1.960 1.960 
Upper Limit 0.203 0.148 0.114 0.096 0.082 0.070 
Lower Limit 0.140 0.116 0.094 0.079 0.067 0.055 
Table 7. 99% Confidence Interval for the Mean of the Ranges 
Percent Complete 
5% 10% 20% 30% 40% 50% 
Number of Contracts 103 141 180 181 181 181 
Mean of the Ranges 0.172 0.132 0.104 0.088 0.075 0.063 
Standard Deviation 0.164 0.097 0.066 0.058 0.053 0.052 
Z Critical Value 25370 2i576/ 2.576. 22576) 25576) “23576 
Upper Limit 0.213 00:53. O60 0.099. "0.085. 0.073 
Lower Limit 0.130 0.111 0.091 0.076 0.064 0.053 
The confidence interval calculations for the mean of 
the ranges provide further videnc that the CP Loum 
stabilizes by the 50% complete point. By looking at Table 


7, the on 





defines s 


complete point. 


ly upper 





limit which breaches the 





At the 


tability for the range method, 


10% 


complete point, 


occurs 


.20 range, 





which 


at the 5% 


the upper limit 


of the 99% confidence interval for the mean of the ranges is 


.153, which 


is well 





below our limit. 
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The following plot of the 99% confidence interval for 
the mean of the ranges illustrates how the interval narrows 


as the contract progresses and the CPlem stabilizes. The 





larger intervals at the lower percent complete points are 





due to the high variances in the CPIlem values during the 





early stages of a contract. The narrower intervals at the 


higher percent complete points show that there is less 





variance in the CPIem values as the contract progresses. 


This narrowing provides evidence that as contracts progress 








the CPlIeum values are becoming more stable. 


99% Confidence Interval for the Mean of the Ranges 








—e— Mean 





—a— Lower Limit 











—»— Upper Limit 
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Figure 1. 99% Confidence Interval for the Mean of the 
Ranges 
Cc. LEAST SQUARES METHOD 


The least squares method was used to identify trends in 


the cost performance. The following four tables show the 





results of the least squares method. Table 8 presents the 





results from using the range method to determine stability 


while Table 9 presents the results from using the narrowing 
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interval method to determine stability. Tables 10 and 11 


show the results from using the narrowing interval method on 








stable and unstable contracts, respectively. The column 
headings are the same for all tables. The first column 
lists the stabilization points observed. These 


Stabilization points should be taken as the earliest 





complete point at which the CPlam stabilized for a given 





contract. The second column identifies the number of 











contracts which stabilized at the specified percent complete 








point listed in the first column. The third and fourth 


columns list the number of contracts which stabilized at the 





specified percent complete point displaying a positive or 





negative slope respectively. The next two columns report 





the maximum and minimum slopes observed, while the final 


column identifies the mean of the slopes. A negative mean 





of least squares depicts a downward trend while a positive 
mean of least squares depicts an upward trend. The value of 
the mean of least squares provides the magnitude of the 


slope. 
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Table 8. 








Least Squares Results for all Contracts using 


Range Method to 


Stabilization 
Point 
70 1 
60 5 
50 0 
40 1 
30 7 
20 46 
10 42 
5 719 
Total 181 
Table 9. 








Define Stability 


Slope 





Freq. Total Total 


(+) (—) 


0 

1 

0 

0 

2 

13 33 
9 
5 
0 








Max. Min. Mean 

Oo 01-5: -=0..015: =0..02-5 
0.160 -0.479 -0.186 
0.242 -0.242 -0.242 
0.336 -0.763 —-0.175 
0.303 -0.472 -0.073 
O259° =—0..242 -—O04.0L3 
0.164 -0.213 -0.026 
0.336 -0.763 -0.046 














Least Squares Results for all Contracts using 





Narrowing 
Stabilization 
Point 
95 1 
90 2 
80 4 
70 4 
60 6 
50 13 
40 LS 
30 19 
20 40 
10 31 
5 46 
Total 181 





Interval Method to 





Define Stability 





Slope 





Freq. Total Total 


a (=) 











1 0 
1 1 
0 4 
1 S 
0 6 
iS 8 
4 11 
4 IS 
13 27 
12 19 
19 27 
60 121 
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Max. Min. Mean 

0.160 0.160 0.160 
0.129 -0.046 0.041 
0.015 -0.479 -0.281 
0.070 -0.154 —-0.054 
0-050 =O.2 702 —Os153 
O.. 3038 -—O027,63% —O0:2106 
0.336 -0.472 -0.098 
0.204 -0.347 -0.107 
0.206 -0.300 —-0.027 
0.156 -0.113 -0.006 
0.105 -0.098 —-0.005 
0.336 -0.763 -0.046 























Table 10. Least Squares Results for Stable Contracts using 
Narrowing Interval Method to Define Stability 





































































































Slope 
Stabilization Freq. Total Total Max. Min. Mean 
Point (+) (—) 
80 3 0 3 -0.269 -0.479 -0.369 
70 2 0 2 -0.042 -0.154 -0.098 
60 3 0 3 -0.050 -0.203 -0.106 
50 7 2 5 0.303 -0.763 -0.124 
40 12 4 8 0.336 -0.472 -0.068 
30 15 3 12 0.143 -0.347 -0.129 
20 30 9 21 0.186 -0.300 -0.031 
10 21 8 13 0. LO6 =Or.013° =04/0.05 
5 38 LS 23 0.105 -0.098 -0.005 
Total 131 41 90 0.336 -0.763 -0.049 
Table 11. Least Squares Results for Unstable Contracts using 
Narrowing Interval Method to Define Stability 
Slope 
Stabilization Freq. Total Total Max. Min. Mean 
Point (+) (=) 
95 1 1 0 0.160 0.160 0.160 
90 2 1 1 0.129 -0.046 0.041 
80 1 0 1 =0':01'5.=0.015° —0.015 
70 2 1 1 0.070 -0.088 —-0.009 
60 3 0 3 -0.066 -0.271 -0.200 
50 6 3 3 O.212 =0.317 -—0:.085 
40 3 0 3 -0.154 -0.280 -0.222 
3:0 4 1 3 0.204 -0.167 -0.025 
20 10 4 6 0.206 -0.121 -0.014 
10 10 4 6 0.053 -0.052 -0.007 
5 8 4 4 0.048 -0.091 -0.007 
Total 50 19 oak 0.212 -0.317 -0.039 
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following observations are made from the tables 


The means of the slopes are negative for all but 
the 90 and 95 percent complete stabilization 
points in Tables 9 and 11. These positive slopes 
are due to the contract baseline being adjusted 
for the two contracts shown to have positive 
slopes. When the baseline is adjusted, the BAC is 
increased to reflect the current overrun. At the 
time when the baseline is adjusted, the BCWPoum is 
set to equal the ACWPoun. Essentially, the CPleun 
is reset to 1.0 at that point in time, thus 
presenting the appearance of a positive trend. 
From Table 10, which excludes all of the unstable 
contracts, it is seen that the means of the slopes 
are negative for all percent complete 
stabilization points. 





























From Table 9, it can be seen that the number of 
contracts having negative slopes was 121 compared 
to only 60 having positive slopes. Thus, there is 
a 67% chance that a contract’s CPlem will decline 
as the contract progresses. 














D. NARROWING INTERVAL METHOD 


The 


results from the narrowing interval method are 


displayed in Table 12. For a contract to be declared stable 





for the percent complete point specified, all CPlIeam values 





must fall on or within the interval produced by subtracting 








.10 from the maximum CPIem value and adding .10 to the 





minimum CPIcoum value. The table reports the number of 





contracts analyzed, the number of those contracts that are 








declared stable, and the associated percentage of contracts 





declared stable. 
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Table 12. Summary of the Narrowing Interval Method CPIoum 
Results 


Percent Complete 



































5% 10% 20% 30% 40% 50% 
Number of Contracts 103 14] 180 181 8 181 
Number Stable 46 77 117 136 15] 164 
% Stable 45% 55% 65% 15% 83% 91% 


In comparison to the range method which showed 








stability down to the 20% complete point, stability occurs 





at the 40% to 50% complete point using the narrowing 


interval method. This is due to the more stringent 








definition of stability, which is that the CPlIleam is within 








plus or minus .10 for all points within th specified 
interval. At the 30% complete point, only 75% of the 





contracts in this study wer declared stable using the 








narrowing interval method compared to 96% using the range 


method. 


E. CONFIDENCE INTERVAL FOR THE PROPORTION OF STABLE 
CONTRACTS USING THE NARROWING INTERVAL METHOD 


The proportion of stable contracts was presented in the 





previous section. In this section, the 90%, 95%, and 99% 
confidence intervals for the proportion of stable contracts 
will be presented. Tables 13 through 15 show the calculated 
confidence intervals for the 90%, 95% and 99% confidence 
levels respectively. The tables list the number of 


contracts analyzed, the number of stable contracts, the 





proportion of stable contracts, the two-tail z critical 





value, and the calculated upper and lower limits. 
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Table 13. 90% Confidence Interval for the Proportion of 
Stable Contracts 


Percent Complete 













































































5% 10% 20% 30% 40% 50% 
Number of Contracts 103 141 180 181 18] 181 
Number Stable 46 77 eg 136 15] 164 
Percent Stable 45% 55% 65% 15% 83% 91% 
Z Critical Value 1.645 1.645 1.645 1.645 1.645 1.645 
Upper Limit 0.528 0.614 0.706 0.800 0.875 0.936 
Lower Limit 0.368 0.477 0.590 0.695 0.784 0.864 

Table 14. 95% Confidence Interval for the Proportion of 
Stable Contracts 
Percent Complete 

5% 10% 20% 30% 40% 50% 
Number of Contracts 103 141 180 181 18] 181 
Number Stable 46 77 117 136 15] 164 
Percent Stable 45% 55% 65% 15% 83% 91% 
Z Critical Value 1.960 1.960 1.960 1.960 1.960 1.960 
Upper Limit 0.543 0.626 0.716 0.809 0.881 0.941 





Lower Limit 0.354 0.464 0.578 0.684 0.773 0.855 





Table 15. 99% Confidence Interval for the Proportion of 
Stable Contracts 





Percent Complete 



































5% 10% 20% 30% 40% 50% 
Number of Contracts 103 14] 180 181 18] 181 
Number Stable 46 77 117 136 15] 164 
Percent Stable 45% 55% 65% 15% 83% 91% 
Z Critical Value 2.576 2.576 2.576 2.576 2.576 2.576 
Upper Limit OL 572 0.65 0.735 0.824 0.893 0.948 





ct CT 


Lower Limit 0.327 0.438 0.555 0.661 0.752 0.835 
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From Table 15, the lower limit for the 99% confidence 
interval at the 40% complete point is only 75%. Thus, it is 


only with reservation that it can be said that stability 











occurs at the 40% complete point. At the 50% complete 





point, the 99% confidence interval, which is from 
approximately 84% to 95%, provides a level of security that 


the CPlom has stabilized. 





F. CATEGORICAL EVALUATION 


The results of the categorical examination are 


presented in Tables 16 through 21. This examination looks 





at the relationship between CPlIem stability and contract 























characteristics. The general characteristics are type, 
phase, time period, and stability of the baseline. The 
specific characteristics for type are: FFP and FFP 
combinations, FPIF, CPFF, and CPIF & CPAF. For phase and 
time period, the specific characteristics are: Development, 
LRIP, and Production; and Pre-Al2, Transitional, and Post- 











Al2 respectively. Lastly, stability of the baseline will be 








classified as stable or unstable. A total of 58 different 





combinations of contract groupings will be evaluated. Not 


all possible combinations are presented; combinations with 





fewer than eight contracts are omitted. The 58 combinations 
are listed as the first column of each table. The second 
column lists the number of contracts in the study with the 


specified characteristics. The remaining two columns report 





the number and percentage of contracts with th specified 





characteristics that have stable CPIgm values for the 





completion point declared in the table title. A separate 





table is presented for the 5%, 10%, 20%, 30%, 40%, and 50% 
complete points. 
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Table 16. Contract Characteristics and Stability 
Relationship from the 5% Complete Point 






















































































Characteristics Number of Number with Percentage with 
Contracts Stable CPLeoum Stable CPIloum 
FFP 0 7 70% 
FPIF 62 27 44% 
CPFF 3 27% 
CPIF 5 7 471% 
Dev 2 8 38% 
LRIP 2 4 33% 
Prod 64 3:1: 48% 
Pre 30 12 40% 
Trans 29 15 52% 
Post 44 19 43% 
S LS 38 52% 
U 30 8 27% 
FPIF/Prod 51 23 45% 
CPFF/Dev 9 2 22% 
CPIF/Dev 8 4 50% 
FPIF/Pre 5 5 33% 
FPIF/Trans 25 12 48% 
FPIF/Post 22 10 46% 
CPFF/Pre 8 2 25% 
CPIF/Post 10 3 30% 
FFP/S 10 7 710% 
FPIF/S 44 21 48% 
FPIF/U 18 6 33% 
Dev/Pre 12 5 42% 
LRIP/Post 9 4 44% 
Prod/Pre 15 7 47% 
Prod/Trans 24 14 58% 
Prod/Post 25 10 40% 
Dev/S Ie 7 54% 
Dev/U 8 1 13% 
LRIP/S 12 4 33% 
Prod/s 44 25 57% 
Prod/U 20 6 30% 
Pre/S 22 11 50% 
Pre/U 8 1 13% 
Trans/S 21 1] 52% 
Trans/U 8 4 50% 
Post/S 30 16 53% 
Post /U 14 3 21% 
FPIF/Prod/S 35 17 49% 
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Characteristics Number of Number with Percentage with 
























































Contracts Stable CPIcoum Stable CPIeoum 
FPIF/Prod/U 16 6 38% 
FPIF/Pre/S 11 5 46% 
FPIF/Trans/S 17 8 47% 
FPIF/Trans/U 8 4 50% 
FPIF/Post/S 16 8 50% 
Prod/Pre/Ss 11 7 64% 
Prod/Trans/S Ly 10 59% 
Prod/Post/S 16 8 50% 
Prod/Post/U 9 2 22% 
FFP Firm Fixed Price and FFP Combinations 
FPIF Fixed Price Incentive F 
CPFF Cost Plus Fixed Fee 
CPIF Cost Plus Incentive Fee & Cost Plus Award Fee 
Dev Development 
LRIP Low Rate Initial Production 
Prod Production 
Pre Pre-Al2 
Trans Transitional] 
Post Post-Al2 
S Stable Baseline 
U Unstable Baseline 
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Table 17. Contract Characteristics and Stability 
Relationship from the 10% Complete Point 































































































Characteristics Number of Number with Percentage with 
Contracts Stable CPloum Stable CPloum 
FFP 5 4 93% 
FPIF 81 42 52% 
CPFF 16 6 38% 
CPIF 23 10 44% 
Dev 30 3 43% 
LRIP 14 oi 50% 
Prod 86 51 59% 
Pre 45 21 47% 
Trans 38 22 58% 
Post 58 34 59% 
Ss 100 59 59% 
U 4l 18 44% 
FFP/Prod 11 10 91% 
FPIF/LRIP 8 3 38% 
FPIF/Prod 66 36 55% 
CPFF/Dev Ly, 5 255 
CPIF/Dev ibe 6 50% 
FFP/Post 8 8 100% 
FPIF/Pre 22 8 36% 
FPIF/Trans 33 19 58% 
FPIF/Post 26 15 58% 
CPFF/Pre 13 5 39% 
CPIF/Post 17 6 35% 
FFP/S 15 14 93% 
FPIF/S 58 30 52% 
FPIF/U 23 12 52% 
CPFF/S 12 4 33% 
CPIF/S 10 7 710% 
Dev/Pre 17 8 471% 
Dev/Post 10 5 50% 
LRIP/Post 11 Tf 64% 
Prod/Pre 24 12 50% 
Prod/Trans 32 Sak 66% 
Prod/Post 30 18 60% 
Dev/S 18 9 50% 
Dev/U 12 4 33% 
LRIP/S 14 af 50% 
Prod/s 61 39 64% 
Prod/U 25 12 48% 
Pre/S 36 17 47% 
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Characteristics Number of Number with Percentage with 




























































































Contracts Stable CPIeoum Stable CPIloum 
Pre/U 9 4 44% 
Trans/S 25 15 60% 
Trans/U 3 7 54% 
Post/S 39 27 69% 
Post /U alee) 7 37% 
FPIF/Prod/S 45 24 53% 
FPIF/Prod/U Zi: 12 57% 
FPIF/Pre/S 18 7 39% 
FPIF/Trans/S 21 12 57% 
FPIF/Trans/U 12 7 58% 
FPIF/Post/S 19 11 58% 
Prod/Pre/S 20 1] 55% 
Prod/Trans/S 21 14 67% 
Prod/Trans/U 11 7 64% 
Prod/Post/S 20 14 70% 
Prod/Post/U 10 4 40% 
FFP Firm Fixed Price and FFP Combinations 
FPIF Fixed Price Incentive F 
CPFF Cost Plus Fixed Fee 
CPIF Cost Plus Incentive Fee & Cost Plus Award Fee 
Dev Development 
LRIP Low Rate Initial Production 
Prod Production 
Pre Pre-Al2 
Trans Transitional] 
Post Post-Al2 
5 Stable Baseline 
U Unstable Baseline 
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Table 18. Contract Characteristics and Stability 
Relationship from the 20% Complete Point 







































































Characteristics Number of Number with Percentage with 
Contracts Stable CPloum Stable CPloum 
FFP 20 19 95% 
FPIF 98 63 64% 
CPFF 24 1.3, 54% 
CPIF 32 17 53% 
Dev 41 24 59% 
LRIP 16 10 63% 
Prod 103 72 710% 
Pre 67 43 64% 
Trans 43 27 63% 
Post 70 47 67% 
Ss 130 89 69% 
U 50 28 56% 
FFP/Prod 13 3 100% 
FPIF/LRIP 8 5 63% 
FPIF/Prod 78 53 68% 
CPFF/Dev 16 9 56% 
CPIF/Dev 18 10 56% 
FFP/Pre 9 8 89% 
FFP/Post 8 8 100% 
FPIF/Pre 32 18 56% 
FPIF/Trans 36 23 64% 
FPIF/Post 30 22 73% 
CPFF/Pre 20 12 60% 
CPIF/Post 25 12 48% 
FFP/S 19 19 100% 
FPIF/S 69 44 64% 
FPIF/U 29 19 66% 
CPFF/S 19 10 53% 
CPIF/S 18 12 67% 
Dev/Pre 22 16 713% 
Dev/Post alg) 8 53% 
LRIP/Post 12 8 67% 
Prod/Pre 35 23 66% 
Prod/Trans 34 24 71% 
Prod/Post 34 25 T4% 
Dev/S 28 18 64% 
Dev/U 13 6 46% 
LRIP/S iis) 10 67% 
Prod/s 74 53 72% 
Prod/U 29 19 66% 
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Characteristics Number of Number with Percentage with 










































































Contracts Stable CPIcoum Stable CPIloum 
Pre/S 55 37 67% 
Pre/U 12 6 50% 
Trans/S 27 18 67% 
Trans/U 16 9 56% 
Post/S 48 34 71% 
Post /U 22 13 59% 
FPIF/Prod/S 54 34 63% 
FPIF/Prod/U 24 19. 19% 
FPIF/Pre/S 27 16 59% 
FPIF/Trans/S BA, 14 67% 
FPIF/Trans/U ab 9 60% 
FPIF/Post/S 21 14 67% 
FPIF/Post/U 9 8 89% 
Prod/Pre/Ss 30 21. 710% 
Prod/Trans/S 22 15 68% 
Prod/Trans/U 12 9 15% 
Prod/Post/S 22 17 17% 
Prod/Post/U 12 8 67% 
FFP Firm Fixed Price and FFP Combinations 
FPIF Fixed Price Incentive F 
CPFF Cost Plus Fixed Fee 
CPIF Cost Plus Incentive Fee & Cost Plus Award Fee 
Dev Development 
LRIP Low Rate Initial Production 
Prod Production 
Pre Pre-Al2 
Trans Transitional 
Post Post-Al2 
S Stable Baseline 
U Unstable Baseline 
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Table 19. Contract Characteristics and Stability 
Relationship from the 30% Complete Point 

























































































Characteristics Number of Number with Percentage with 
Contracts Stable CPloum Stable CPloum 
FFP 20 ig 95% 
FPIF 99 wos) TAS 
CPFF 24 Ly 71% 
CPIF 32 Zs 66% 
Dev 41 27 66% 
LRIP Li] 15 88% 
Prod 103 80 718% 
Pre 67 49 713% 
Trans 43 29 67% 
Post Hai 58 82% 
Ss 131 104 19% 
U 50 32 64% 
FFP/Prod 13 I3 100% 
FPIF/LRIP 9 9 100% 
FPIF/Prod 78 58 TAS 
CPFF/Dev 16 al 69% 
CPIF/Dev 18 ats 61% 
FFP/Pre 9 8 89% 
FFP/Post 8 8 100% 
FPIF/Pre 32 20 63% 
FPIF/Trans 36 25 69% 
FPIF/Post 31 28 90% 
CPFF/Pre 20 16 80% 
CPIF/Post 25 16 64% 
FFP/S 19 19 100% 
FPIF/S 70 52 TAS 
FPIF/U 29 21 12% 
CPFF/S 19 14 T4% 
CPIF/S 18 14 718% 
Dev/Pre 22 18 82% 
Dev/Post 15 9 60% 
LRIP/Post ales: 12 92% 
Prod/Pre 35 24 69% 
Prod/Trans 34 26 717% 
Prod/Post 34 30 88% 
Dev/S 28 Zl 715% 
Dev/U 13 6 46% 
LRIP/S 16 eS 94% 
Prod/s 74 58 718% 
Prod/U 29 22 716% 
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Characteristics Number of Number with Percentage with 













































































Contracts Stable CPIcoum Stable CPIloum 
Pre/S 55 42 716% 
Pre/U 12 7 58% 
Trans/S 27 19 70% 
Trans/U 16 10 63% 
Post/S 49 43 88% 
Post /U 22 1S 68% 
FPIF/Prod/S 54 38 710% 
FPIF/Prod/U 24 20 83% 
FPIF/Pre/S 27 17 63% 
FPIF/Trans/S DA, 15 71% 
FPIF/Trans/U 15 10 67% 
FPIF/Post/S 22 20 91% 
FPIF/Post/U 9 8 89% 
Prod/Pre/Ss 30 22 713% 
Prod/Trans/S 22 16 73% 
Prod/Trans/U 12 10 83% 
Prod/Post/S 22 20 91% 
Prod/Post/U 12 10 83% 
FFP Firm Fixed Price and FFP Combinations 
FPIF Fixed Price Incentive F 
CPFF Cost Plus Fixed Fee 
CPIF Cost Plus Incentive Fee & Cost Plus Award Fee 
Dev Development 
LRIP Low Rate Initial Production 
Prod Production 
Pre Pre-Al2 
Trans Transitional 
Post Post-Al2 
S Stable Baseline 
U Unstable Baseline 
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Table 20. Contract Characteristics and Stability 
Relationship from the 40% Complete Point 

























































































Characteristics Number of Number with Percentage with 
Contracts Stable CPloum Stable CPloum 
FFP 20 19 95% 
FPIF 99 86 87% 
CPFF 24 18 715% 
CPIF 32 22 69% 
Dev 4l 28 68% 
LRIP Li] 1S. 88% 
Prod 103 94 91% 
Pre 67 56 84% 
Trans 43 34 19% 
Post Hai 61 86% 
Ss 131 116 89% 
U 50 35 710% 
FFP/Prod 13 13 100% 
FPIF/LRIP 9 9 100% 
FPIF/Prod 78 71 91% 
CPFF/Dev 16 12 75% 
CPIF/Dev 18 i 61% 
FFP/Pre 9 8 89% 
FFP/Post 8 8 100% 
FPIF/Pre 32 26 81% 
FPIF/Trans 36 30 83% 
FPIF/Post 3:1. 30 97% 
CPFF/Pre 20 17 85% 
CPIF/Post 25 17 68% 
FFP/S 19 19 100% 
FPIF/S 70 63 90% 
FPIF/U 29 23 19% 
CPFF/S 19 15 19% 
CPIF/S 18 14 718% 
Dev/Pre 22 19 86% 
Dev/Post 15 9 60% 
LRIP/Post ales: 12 92% 
Prod/Pre 35 30 86% 
Prod/Trans 34 31 91% 
Prod/Post 34 33 97% 
Dev/S 28 22, 19% 
Dev/U 13 6 46% 
LRIP/S 16 IS 94% 
Prod/s 74 69 93% 
Prod/U 29 25 86% 
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Characteristics Number of Number with Percentage with 













































































Contracts Stable CPIeoum Stable CPIloum 
Pre/S 55 49 89% 
Pre/U 12 7 58% 
Trans/S 27 22 81% 
Trans/U 16 le? 15% 
Post/S 49 45 92% 
Post /U 22 16 713% 
FPIF/Prod/S 54 49 91% 
FPIF/Prod/U 24 22 92% 
FPIF/Pre/S 27 23 85% 
FPIF/Trans/S DA, 18 86% 
FPIF/Trans/U 15 12 80% 
FPIF/Post/S 22 22 100% 
FPIF/Post/U 9 8 89% 
Prod/Pre/S 30 28 93% 
Prod/Trans/S 22 19 86% 
Prod/Trans/U 12 12 100% 
Prod/Post/S 22 22 100% 
Prod/Post/U 12 11 92% 
FFP Firm Fixed Price and FFP Combinations 
FPIF Fixed Price Incentive F 
CPFF Cost Plus Fixed Fee 
CPIF Cost Plus Incentive Fee & Cost Plus Award Fee 
Dev Development 
LRIP Low Rate Initial Production 
Prod Production 
Pre Pre-Al2 
Trans Transitional 
Post Post-Al2 
S Stable Baseline 
U Unstable Baseline 
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Table 21. Contract Characteristics and Stability 
Relationship from the 50% Complete Point 













































































Characteristics Number of Number with Percentage with 
Contracts Stable CPloum Stable CPloum 
FFP 20 19 95% 
FPIF 99 90 91% 
CPFF 24 22 92% 
CPIF 32 27 84% 
Dev 4l 35 85% 
LRIP Li] 1S. 88% 
Prod 103 98 95% 
Pre 67 59 88% 
Trans 43 39 91% 
Post 71 66 93% 
Ss 1833 123 94% 
U 50 4l 82% 
FFP/Prod 13 153 100% 
FPIF/LRIP 9 9 100% 
FPIF/Prod 78 74 95% 
CPFF/Dev 16 14 88% 
CPIF/Dev 18 16 89% 
FFP/Pre 9 8 89% 
FFP/Post 8 8 100% 
FPIF/Pre 32 27 84% 
FPIF/Trans 36 33 92% 
FPIF/Post 3:1. 30 97% 
CPFF/Pre 20 19 95% 
CPIF/Post 25 21 84% 
FFP/S 19 19 100% 
FPIF/S 70 66 94% 
FPIF/U 29 24 83% 
CPFF/S 19 17 90% 
CPIF/S 18 16 89% 
Dev/Pre 22 19 86% 
Dev/Post 5 14 93% 
LRIP/Post ales: 12 92% 
Prod/Pre 35 32 91% 
Prod/Trans 34 33 97% 
Prod/Post 34 33 97% 
Dev/S 28 25 89% 
Dev/U 13 10 717% 
LRIP/S 16 iS 94% 
Prod/s 74 ee 97% 
Prod/U 29 26 90% 
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Characteristics Number of Number with 
Contracts Stable CPI 

Pre/S 55 aval 
Pre/U 12 8 
Trans/S 27 25 
Trans/U 16 14 
Post/S 49 47 
Post /U 22 19 
FPIF/Prod/S 54 52 
FPIF/Prod/U 24 22 
FPIF/Pre/S 27 24 
FPIF/Trans/S DA, 20 
FPIF/Trans/U 15 13 
FPIF/Post/S 22 22 
FPIF/Post/U 9 8 
Prod/Pre/S 30 29 
Prod/Trans/S 22 21 
Prod/Trans/U 12 12 
Prod/Post/S 22 D2 
Prod/Post/U Aliee, sl 
FFP Firm Fixed Price and FFP Combinations 
FPIF Fixed Price Incentive F 
CPFF Cost Plus Fixed Fee 
CPIF Cost Plus 
Dev Development 
LRIP Low Rate Initial Production 
Prod Production 
Pre Pre-Al2 
Trans Transitional 
Post Post-Al2 
S Stable Baseline 
U Unstable Baseline 

The study of the relationship 







































































cum Stable CP Toum 


between 


93% 
67% 
93% 
88% 
96% 
86% 
96% 
92% 
89% 
95% 
87% 
100% 
89% 
97% 
96% 
100% 
100% 
92% 





the 


Incentive Fee & Cost Plus Award Fee 


CPI 





Percentage with 


cum 


stabilization points and the contract characteristics allows 


comparison among the different characteristics. 


the four types 


of contracts, 


percentage of stability than the other types. 
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Comparing 


FFP contracts have a higher 


From Table 


17, 93% of the FFP contracts were stable at the 10% complete 





point. For the remaining three types, FPIF and CPFF 





contracts have higher percentages of stability compared to 








CPIF & CPAF contracts from the 20% complete point onward. 











It should be noted that in general, fixed price contracts 
are more stable than incentive and award fee contracts. 
From Table 21, there were 20 FFP contracts and 24 CPFF 
contracts, of which, 19 and 22 were stable at the 50% 
complete point respectively. Combining these two types, 41 


out of 44, or 93% of the fixed price contracts are stable at 








the 50% complete point. In comparison, there were 99 FPIF 








contracts and 32 CPIF contracts, of which, 90 and 27 were 


stable at the 50% complete point respectively. Combining 





these two types, 117 out of 131, or 89% of the incentive and 


award fee contracts are stable at the 50% complete point. 








As one person in OSD who is knowledgeable about this issue 


said: 


The categorical analysis fits intuitively. FPIF 
and FFP contracts are bid on programs or phases of 
the programs that contractors understand very well 
(less risk). New developments or al Ls 
defined/poorly scoped programs (lots of risk) will 
be bid out as CPIF or CPAF. (LTC J. Thurman, 
personal communication, May 15, 2007) 




















Looking at the different phases examined in this study, 





contracts in the production phase had a higher percentage of 











stability at the 40% and 50% complete points relative to the 





other two. The LRIP contracts had a higher percentage of 








stability compared to development contracts from the 10% 


through the 50% complete point. 








Contracts started after the A12 program cancellation 





had a higher percentage of stability relative to contracts 
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started before cancellation. Although th percentag is 


only slightly higher, this provides som vidence that the 








EVMS changes influenced by the Al12 program cancellation may 


be having a positive impact. 





Lastly, comparing contracts with stabl baselines to 











those with unstabl baselines, it is clear that contracts 








with stable baselines have a higher percentag Of CP Leum 





stability. For all complete points presented, contracts 





with stable baselines outperformed those with unstable 


baselines. 


The relative findings just identified for type, phase, 





and baseline stability are in agreement with the findings 
presented by Heise (1991). The final chapter will provide a 


summary of all the findings presented in this chapter, as 





well as discuss the significance of the findings and 


recommend ideas for further research. 
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V. SUMMARY AND DISCUSSION 


A: REVIEW OF THE HYPOTHESIS 





The null hypothesis is that the CPIgm stabilizes by the 











50% complete point of a contract. The hypothesis was tested 





using both the range method and the narrowing interval 








method to determine CPlem stability. Both methods were 


applied on a sample of 181 contracts from 48 different 








programs taken from the DAMIR system. 


B. CONCLUSION 





The range method shows that the CPIem stabilized by the 





50% complete point for 97% of the contracts included in this 








study. In fact, evidence was provided that the CPlIem began 
to stabilize from the 10% to 20% complete points. In Table 








7, the upper limits for the 99% confidence intervals at the 
10% and 20% complete points were .153 and .117 respectively, 
which are both well within the .20 limit defined by the 


range method. 





For the narrowing interval method, it was shown that 





the CPlem stabilized for 91% of the contracts included in 








this study by the 50% complete point. From Table 15, the 


lower limit of the 99% confidence interval at the 40% 





complete point was down to .752; thus stability will only be 





declared for contracts at the 50% complete point or greater. 
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Cc. DISCUSSION 








Declaring the CPIeam stable is not an end in itself. 





One of the many benefits of the CPlIem is its value in 





estimating the cost at completion. The CPlIeum is only useful 


for this purpose if it can be declared stable. By the 











definition of CPlIleam stability, which is that CPlIeum will not 





vary by more than plus or minus .10, estimates made using a 





stable CPleum will not be more than 10% off the final cost of 








the contract. The definition just stated for CPleoum 


stability is the one used in the narrowing interval method. 





Thus, the narrowing interval method provides a better 





measure of when a contract’s CPleam should be declared 


stable. 


This study provides PMs and analysts with a _ solid 





foundation of CPlIem stability percentages. This stability 








is observed in historical contracts at varying complete 





points and for various contract categories. With the 
results of this study, PMs and analysts should have more 


confidence in their cost at completion estimates. 


The results can also be used to determin th 





probability that a contractor can recover from a cost 




















overrun. In Chapter , the To Complete Performance Index 








(TCPI) was defined as how well the contractor must perform 


throughout the remainder of the contract to finish within 











the planned budget. If the CPlIleum can be declared stable for 





a contract and the calculated TCPI is .10 or more above the 











current CPIoum value, then by definition of CPIgm stability, 


the contractor will not be able to complete the contract 





within the planned budget. Further, from the method of 


56 





least squares, it was shown that the CPIgym, for the average 


contract tends to decline rather than improve. 











An example is provided below to demonstrate the utility 











of the results of this study. A FPIF contract, in the 








production phase, and with a stable baseline, has the 


following cost performance data: 























Table 22. Cost Performance Data for Fictional Contract 
(SMillions) 
BCWP cum ACWP cum BAC 
100 110 250 
Using equation (6), the percent complete is found by 





dividing the BCWPom by the BAC, which is .4 [=100 / 250]. 





The table providing the contract characteristics and 


stability relationship from the 40% complete point is then 








checked to determine stability. From Table 20, the row 


labeled “FPIF/Prod/S” shows that 91% of the contracts 





reviewed in this study with the same characteristics as this 





fictional contract had stable CPIem values at the 40% 








complete point. It is thus assumed that the CPlIum to be 


calculated for this contract may be declared stable. From 





equation (4), the CPloum is calculated by dividing the BCWPoun 





by the ACWPom, which is .909 [=100 / 110]. Equation (3) is 











now used to calculate the TCPI. The TCPI is found by 
dividing the BAC minus the BCWPeam by the BAC minus the 
ACWPoumy Which is 1.07 [=(250 - 100) / (250 - 110)]. From 





our definition of stability, the maximum that the CPleuwm can 





possibly improve to is 1.009 [=.909 + .10]. Thus, it is 
highly unlikely that the contractor will be able to finish 








within the planned BAC. Incorporating the results from the 


Sef 


method of least squares, the contractor will not only be 


unable to recover, it is highly likely that performance will 





decline throughout the remainder of the contract. At this 








point, the CPIgym could further b used to determine an 





accurate estimate at completion. 


D. RECOMMENDATIONS FOR FURTHER RESEARCH 


This study looked at contracts which had cost 








performance data through the 85 percent complete point. It 
is recommended that historical data be analyzed for 
contracts which did not make it to the 85 percent complete 
point due to cancellation for poor cost performance. This 
would provide PMs and analysts with information on the early 
trends exhibited by contracts which are susceptible to 


cancellation. 











Another area to investigate within the EVMS is the SPI. 


Many of the heuristics used to determin th FAC use a 





combination of the SPloem and the CPlIlocun. The SPleum by 











definition (comparison of the cumulative budget for work 


actually performed with the cumulative budget for tasks 








scheduled) starts at 1.0 and returns to DQ at the 





completion of the contract. It is recommended that a 





historical analysis be conducted to determine trends in the 











SPleum in comparison to contract performance. For instance, 





what is the average time for schedul recovery when the 


SPleum is .90? For cancelled programs, what is the average 





SPlewm at specified percent complete intervals? As the 





recommended study above suggests, this would provide PMs and 
analyst with trends displayed by contracts which have been 


terminated. 
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APPENDIX A: 


Table 23. 


Contracts Listed by Name, 


Phase, 


Stability of the Baseline 


Program/Contract 


AIM-9X 
AIM-9X 
AIM-9X 
AIM-9X 

AN/BSY-1 
TT/WLSOT 

AN/SQQ-89 
EMSP S/W Conversion 








Dev/Prod 





Shipboard Elect Subsys. 





Array Equipment 

















AOE-6 
AOE 10 
AOE 8 
AOE 7 
ASPJ 
ASPJ Lot I Prod 
ASPJ Lot I Production 
C/MH-53E 
FY78 Buy 6 A/C 
FY80 Buy 15 A/C 
FY79 Buy 14 A/C 
CAPTOR (MK 60 MINE) 
Mine 
CG-47 


CG 69/71-73 Constr. 
CG-70 Constr. 

CG 66/8 Constr. 
Aegis Wpn Sys 

CG 67 Constr. 

CG 62/5 Constr. 

CG 60/1/3/4 Constr. 


CG-48 (Yorktown) 
CG-47 (Ticonderoga) 
CVN 68 


CVN 76 Constr. 
CVN-74/75 Constr. 
DDG 1000 (DD (xX) ) 














CP 
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Type 


IF/CPAF 





CPIF 
CPIF 





FPIF 








Phase 


Dev 
Dev 
Dev 





Prod 
Prod 


Prod 
Prod 
Prod 


Prod 
Prod 
Prod 
Prod 
Prod 
Prod 
Prod 
Prod 
Prod 


Prod 
Prod 


Type, 


CONTRACTS INCLUDED IN STUDY 






















































































Year, and 

Year S/U 
1997-2003 S 
1995-1996 S 
1995-1996 S 
1988-1991 S 
1993-1995 S 
1980-1983 S 
1980-1983 S 
1993-1997 S 
1991-1995 U 
1990-1994 U 
1989-1992 S 
1990-1992 S 
1978-1982 S 
1981-1982 S 
1979-1982 U 
1981-1983 S 
1989-1993 S 
1988-1993 S 
1987-1992 S 
1988-1992 S 
1987-1992 S 
1987-199] S 
1987-199] U 
1981-1984 S 
1979-1983 U 
1995-2003 S 
1988-1998 U 


DD (X) 
DDG 51 
BO 7 91 93a: 
DG90/ 92... 
egis Wpn 














egis Wpn 


egis Wpn 
DG 69,71 





egis Wpn 
DG 55,57 


egis Wpn 


egis Wpn 


OUPUPVUVUUPUVUVUVUUPUVUPVVGVUS UV 








Aegis Wpn 


Phase 


DG 83/85/87 Con 
DG 84/86/88 Con 
DG 78,80, 
DG 77,79, 


DG 73,75, 
DG 74 Constr. 




















Constr. 
Constr. 
sys 





82 Con 
81 Con 
sys 


ee ae 








Dev 


76 Constr. 


sys 
Constr. 


63... Const 
64... Const 
sys 
Constr. 


sys 


DG 52 Constr. 


sys 





DDG 51 Constr. 


sys 


DG 68,70,72 Constr. 
DG 59,61, 
DG 60, 62, 


Cr. 
tw. 





DG 54,56,58 Constr. 
DG 53 Constr. 


Combat Sys Engineering 
E-2C REPRODUCTION 








EFV 





Dem/Val 
F-14D 








FY8Q Buy 30 A/C 
Airframe Prod 


Airframe 


F/A-18A/B/C/D 





Airframe |] 
Engine 
F/A-18E/F 





Dev 


Airframe LRIP 3 





Engine LRI 





P 2/3 








Engine EMD 
Airframe EMD 





Airframe LRIP 2 





SSIPS 








Msn Computer Upgrade 


CPI 


CPAF 


FPI 
FPI 
FPI 
FPI 
FPI 
FPI 
EP T 
EPI. 
FPI 
FPI 
FPI 
FPI 
FPI 
FPI 
FPI 
FPI 
tea 
FPI 
FPI 
EPI 
FPI 
FPI 
FPI 
FPIF 
CPFF 


RY RY RY YY RY YY RY YY RY RY AY AY AY AY RY RY AY HY AY OF 








F/CPAF 
CPAF 
FFP 


FFP 
FFP 
































































































































Dev 2003-2006 
Prod 1998-2006 
Prod 1999-2006 
Prod 1999-2004 
Prod 1996-2002 
Prod 1996-2002 
Prod 1994-2000 
Prod 1995-2000 
Prod 1997-2000 
Prod 1993-1998 
Prod 1993-1998 
Prod 1994-1998 
Prod 1992-1997 
Prod 1992-1997 
Prod 1991-1996 
Prod 1990-1996 
Prod 1992-1995 
Prod 1989-1994 
Prod 1989-1994 
Prod 1988-1993 
Prod 1990-1993 
Prod 1988-1992 
Prod 1988-1992 
Prod 1985-199] 
Prod 1986-1989 
Dev 1982-1984 

1995-2000 
Dev 1997-2001 
Prod 1980-1982 
Prod VO7T9=1L984 

1977-1978 
Dev 1976-1982 
Dev 1976-198] 
LRIP 1999-2001 
LRIP 1999-2001 
Dev 1992-2000 
Dev 1992-2000 
LRIP 1998-2000 
Dev 992-1996 








NNNNNNAG io) 


C WN C 


nn 








NNnNaQGnN 


G 


FDS UWS FSED 




















FEFG-7 
FY80 Buy (3 Ships) 
FY79 Buy (3 Ships) 
FY78 Buy (3 Ships) 
FY78 Buy (2 Ships) 
FY77 Buy (3 Ships) 
FY78 Buy (3 Ships) 
FY77 Buy (2 Ships) 
Five Ships 
Three Ships 

H-1 UPGRADES (4BW/4BN) 





H-1l Upgrade EMD Constr. 


HARM (NAVY) 
FORD LCS 
LCS Dev §S 

HARPOON 
Missile 

JSOW 








Dev S 





JSOW Unitary E&MD 








JSOW LRIP 














JTIDS (NAVY) 








Full Scale 





LAMPS MK 

















(SH-6 


Airframe Dev 
Sys Integration 
Engine Dev 




















LCAC 
LCAC 61-72 Con 
LCAC 49-51 Con 
LCAC 52-60 Con 
LCAC 37-48 Con 
LCAC 34-36 Con 
LCAC 15-23 Con 
LCAC 24-33 Con 
LHD 1 
LHD 6 Constr. 
LHD 5 Constr. 
LHD 4 Constr. 
LHD 3 Constr. 
LHD 2 Constr. 
LPD 17 
LPD 19 
LPD 18 
LPD 17 








MH-60R 


Dev 


upport 





upport (RAYTH) 





OB) 


tL. 
GL. 
tr. 
ea ae 
tr. 
tr. 
CLs. 





AnNANANDAN YN 


REP RR Ry Ry RY 











RR] R] Ry] 


REY RL A] 
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Ry | 
































































































































Dev 1990-1995 
Prod 1980-1984 
Prod 1980-1984 
Prod 1979-1983 
Prod 1978-1983 
Prod 1979-1983 
Prod 1979=1983 
Prod 1977-1983 
Prod LOTTA LIS | 
Prod 1977-198] 
Dev 1997-2005 
Dev 1985-1988 
Dev 1985-1987 
Prod 1976-1979 
Dev 1996-2003 
LRIP 1998-2000 
Dev 1982-1985 
Dev 1978-1982 
Dev 1977-1982 
Dev 1978-1982 
Prod 1991-1994 
Prod 1990-1993 
Prod 1990-1993 
Prod 1989-1993 
Prod 1989-1992 
Prod 1987-199] 
Prod 1987-199] 
Prod 1993-1998 
Prod 1992-1997 
Prod 1989-1994 
Prod 1988-1993 
Prod 1987-1992 
Prod 2000-2005 
Prod 1999-2005 
Prod 1997-2005 


eG 


NNNNNNNNMN 


NNNNnNNNN 

















Dev (EMD I) 
Dev (ALFS) 
MHC 51 
MHC 61/62 (Option) 
MHC 58,59, &60 
MHC 56/57 (Option) 


MK 48 ADCAP 
L3 Test Equipment 
Follower Pilot (P1) 

MK 50 TORPEDO 
MK-50 Torpedo LRIP 
MK-50 Torpedo LRIP 
MK 50 Torpedo Qual... 
MK 50 Torpedo LRIP 
MK 50 FSED 

PHALANX CIWS (MK-15) 

FY 87 GD Prod 
86 Prod 

PHOENIX (AIM—-54C) 
Guidance Cntrl & AFRM 

SIDEWINDER (AIM-9L) (N) 

















































































































Guidance Dev 

Optical Tgt Detector 
SM 2 (BLKS I-IV) 

SM-2 Blk IV FY95-98... 

SM-2 Blk GC&A 

MK-45 Mod 5 FY 87 Prod 
SSDS 

WASP Minimissile 
SSGN 

Detail Design 








SSN 21 / AN/BSY-2 
SSN 22 Constr. 
SSN 21 Constr. 
AN/BSY-2 LP 
AN/BSY-2 FSD 

















SSN 21 Detail Design 
NNS Contract Design 
SSN 688 





SSN 688 Attack Sub 
Flight X Ships 
SSN 700-710 

SSN 774 (VIRGINIA CLASS) 
SSN775 Constr. 

IPPD96 Contract 

NSSN C31 Prime Contract 





































































































































































































CPFF Dev 1993-1999 
CPIF Dev 1992-1997 
FFP Prod 1993-1998 
FFP Prod 1993-1997 
FPIF Prod 1992-1996 
FPIF 1988-199] 
FPIF 1988-199] 
FPIF LRIP 1990-1992 
FPIF LRIP 1990-1992 
FFP/FPIF LRIP 1989-199] 
FPIF LRIP 1989-199] 
CPIF/FFP Dev 1983-1990 
FPIF Prod 1988-1990 
FPIF Prod 1987-1989 
FPIF Prod 1987-1992 
CPFF Dev 1977-1980 
FPIF Dev 1977-1979 
CPAF/FPIF LRIP 1996-2002 
FPIF Prod 1987-1990 
FFP Prod 1988-1989 
FPIF 1980-1983 
CPIF Dev 2003-2005 
FPIF Prod 1993-1998 
FPIF Prod 1990-1997 
FPIF 1989-1996 
FPIF Dev 1988-1996 
CPFF Dev 1988-1995 
CPFF Dev 1985-1987 
FPIF Prod 1987-1994 
FPIF Prod 1986-1993 
FPIF Prod 1974-1983 
CPIF Prod 1999-2006 
CPFF/CPIF 1996-2004 
CPAF 1996-2000 
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MeWNE WH Ge io) 


nn 











STRATEGIC SEAL 
New Constr. 
New Constr. 
Class Standa 
Conversion 

T-45TS 
Cockpit 21 

T-AKE 
New Constr., 
New Constr., 

TOMAHAWK 
FY81 CWCS Pr 

















Bel 


rd Equip. 


T-AKE 3 
T-AKE 1 


od 





FY82 Cruise 


Engine 


SLCM/GLCM CWCS 














LE 








TRIDENT MISS 
Missile Fol 
Missile Fol 
Missile Foll 
Missile Foll 
Guidance Pie 
Missile Foll 
Missile Fol 
Missile Fol 
Navigation O 
Missile Op. 

Missile Op. 

Test Instr. 

Launcher Op. 
Fire Control 
Guidance Sys 
DENT SUB 

ubmarine Gr 









































TR 

















low-on 
low-on 


low-on 
low-on 


Prod 
Prod 
ow-on Prod 
ow-on Prod 
ce Prod 

ow-on Prod 
Prod 
Prod 





p. sys 
sys 

sys P 

Op. Sys 
sys 
Op. 

Dev 


sys 











p VII Ships 











ubmarine Gr 
ubmarine Gr 








D 
Ss 
Submarine Gr 
S 
S 


p VI Ships 
p V Ships 














TRIDENT SUB 
FY79 Missile 
Missile Prod 
V-22 
FYO4 LRIP Lo 
FYO05 LRIP Lo 
EMD Airframe 
FYO03 LRIP Lo 
FYOO LRIP 4 
FY99 LRIP 3 
CV-22 FFS#1 











p IV Ships 


Prod 
(C-4) 
t 8 Airframe 
9 Airframe 





t 7 Airframe 
Airframe 
Airframe 
& NEWID 
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FPIF Prod 1994-2001] 
FFP Prod 1994-2001] 
FFP/FPAF Prod 1993-2000 
FPIF 1993-1997 
CPIF Dev 1993-1996 
FPIF Prod 2003-2006 
FPIF Prod 2003-2005 
CPFF Prod 1982-1984 
FPIF Prod 1982-1984 
CPFF Prod 1980-1983 
CPIF/FFP Prod 1995-1997 
CPIF/FFP Prod 1994-1997 
CPIF/FFP Prod 1992-1996 
CPIF/FFP Prod 1991-1994 
FPIF Prod 1992-1993 
CPIF/FFP Prod 1990-1993 
CPIF Prod 1989-1992 
CPIF Prod 1988-199] 
CPIF/FPIF Dev 1984-1990 
CPIF/FFP Dev 1983-1990 
CPIF/FFP Dev 1987-1990 
CPIF Dev 1984-1990 
CPIF/FFP Dev 1983-1989 
CPIF Dev 1984-1989 
CPFF Dev 1983-1988 
FPIF Prod 1988-1996 
FPIF Prod 1987-1993 
FPIF Prod 1986-1992 
FPIF Prod 1982-1990 
CPFF Prod 980-198 
CPFF Prod 978-198 
FPIF/CPIF LRIP 2003-2006 
FFP/FPIF LRIP 2005-2006 
CPAF 1993-2006 
FPIF LRIP 2003-2005 
FPIF LRIP 2000-2003 
CPIF LRIP 1998-2003 
CPIF 2000-2003 


























nN Gnwnwn 
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MV-FFS#2 
MV-22 LRIP SIM.FFS/FTD 
FY98 LRIP 2 (Airframe) 
FY97 LRIP 1 (Airframe) 
EMD (Engine) 

Tech Effort 

Prelim Design Stage 
Prelim Design Stage 
















































































CPIF 2001-2003 
CPIF LRIP 1998-2002 
CPIF/CPFF LRIP 1997-2002 
CPIF/CPFF LRIP 1996-2002 
CPIF Dev 1993-1996 
CPFF Dev 1991-1996 
CPFF Dev 1983-1986 
CPFF Dev 1984-1985 
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APPENDIX B: RESULTS OF RANGE METHOD 


Table 24. Range From Given Percent Completion Point to Final 
CPR Entry 


Percent Complete 



































































































































Program/Contract 
5% 10% 20% 30% 40% 50% 
AIM-9X 
AIM-9X 0.08 0.05 0.04 0.03 0.03 
AIM-9X 0.22 0.16 0.14 0.09 
AIM-9X 0.12 Ov 0.09 0.07 
AN/BSY-1 
TT/WLSOT Dev/Prod 0.34 0.34 0.24 0.24 
AN/SQQ-89 
EMSP S/W Conversion 0.05 0.03 0.03 0.02 
Shipboard Elec 
Subsys 0.16 0.08 0.05 0.03 
Array Equipment 0.13 O.11 O.11 0.08 
AOE-6 
AOE 10 0.09 0.09 0.09 O09 O09 O09 
AOE 8 0.2 0.2 0.19 0.09 
AOE 7 0:23 023 O23 O23 
ASPJ 
ASPJ Lot I Prod 0.71 0.42 0.19 O.11 0.09 0.08 
ASPJ Lot I 
Production 1 OS 0.51. 0435 0: sel 0.12 0.05 
C/MH-53E 
FY78 Buy 6 A/C 0.4 0.27 O.1 0.09 0.09 0.07 
FY80 Buy 15 A/C OS 0.08 0.05 0.03 0.02 
FY79 Buy 14 A/C 0.28 0.28 0.28 0.18 0.14 0.14 
CAPTOR (MK 60 MINE) 
Mine 0.06 0.05 0.05 0.05 0.05 
CG-47 
CG 69/71-73 Constr. O.13 0.13 0.13 0.13 0.13 0.13 
CG-70 Constr. 0.08 0.08 0.06 0.05 0.05 0.05 
CG 66/8 Constr. 0.11 0.11 0.11 0.11 0.07 0.07 
Aegis Wpn Sys 0.06 0.06 0.06 0.06 0.06 
CG 67 Constr. O.1 0.1 0.08 0.08 0.08 0.07 
Ce 6275 Constr. 0.09 0.04 0.04 0.04 
CG 60/1/3/4 Constr. 0.04 0.03 0.03 0.02 
CG-48 (Yorktown) 0.05 0.04 0.04 0.02 0.02 
CG-47 (Ticonderoga) O52 D233 0.16 0.16 Ore 1S O.1 




















CVN 68 
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